Gradient calculations for dynamic recurrent neural networks: a survey

Gradient Calculations for Dynamic Recurrent Neural Networks: A Survey

We survey learning algorithms for recurrent neural networks with hidden units and put the various techniques into a common framework. We discuss fixed point learning algorithms, namely recurrent backpropagation and deterministic Boltzmann machines, and nonfixed point algorithms, namely backpropagation through time, Elman’s history cutoff, and Jordan’s output feedback architecture. Forward propa...

Gradient calculations for dynamic recurrent neural networks: a survey

Surveys learning algorithms for recurrent neural networks with hidden units and puts the various techniques into a common framework. The authors discuss fixed point learning algorithms, namely recurrent backpropagation and deterministic Boltzmann machines, and nonfixed point algorithms, namely backpropagation through time, Elman's history cutoff, and Jordan's output feedback architecture. Forwa...

DRAFT OF July 20, 1995FOR IEEE TRANSACTIONS ON NEURAL NETWORKS 1 Gradient Calculations for Dynamic Recurrent Neural Networks: A Survey

A Unifying View of Gradient Calculations and Learning for Locally Recurrent Neural Networks

In this paper a critical review of gradient-based training methods for recurrent neural networks is presented including Back Propagation Through Time (BPTT), Real Time Recurrent Learning (RTRL) and several specific learning algorithms for different locally recurrent architectures. From this survey it comes out the need for a unifying view of all the specific procedures proposed for networks wit...

Dynamic recurrent neural networks

We survey learning algorithms for recurrent neural networks with hidden units and attempt to put the various techniques into a common framework. We discuss fixpoint learning algorithms, namely recurrent backpropagation and deterministic Boltzmann Machines, and non-fixpoint algorithms, namely backpropagation through time, Elman's history cutoff nets, and Jordan's output feedback architecture. Fo...